Medical image processing apparatus

ABSTRACT

A medical image processing apparatus is provided which generates a three-dimensional superposed image showing an inserted medical device in a preoperative image. More specifically, the medical image processing apparatus comprises; an input unit; an image signal generating unit; an image information generating unit which generates image information for acquiring a superposed image having a three-dimensional image of the medical device over a three-dimensional image a preoperative structure; and an output unit.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Divisional of co-pending U.S. patent applicationSer. No. 13/910,947, filed Jun. 5, 2013, which claims foreign prioritybenefit of Japanese Patent Application No. 2012-131118, filed Jun. 8,2012, both which are hereby incorporated by reference herein in theirentirety.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to medical image processing apparatuseswhich accurately visualize an operating environment for a medicaldevice, such as an endoscope, to be inserted to an object.

Description of the Related Art

It is difficult to insert a surgical tool, such as an endoscope and acatheter, having a complicated form to an intended region through aninternal luminal structure such as blood vessel or a body cavity, forexample.

Accordingly, Japanese Patent Laid-Open No. 5-177000 discloses creating amap of an internal luminal structure from an image of an internalluminal structure captured with an X-ray or by CT, MRI or the likebefore an operation and performing navigation for inserting a tip of asurgical tool into an internal desirable position.

For an intraoperative navigation, a position of a tip of a surgical toolmay be detected with an X-ray. Alternatively, a transmitting unit may beattached to the tip of the surgical tool and a signal transmitted fromthe transmitting unit may be received by a receiving unit. The positionof the transmitting means is acquired from the received signal in realtime. The position may be displayed over the map.

According to the method disclosed in Japanese Patent Laid-Open No.5-177000, a two-dimensional map is used. Therefore, what kind ofattitude a surgical tool has may not be known three-dimensionally.

SUMMARY OF THE INVENTION

According to the present invention, there is provided a medical imageprocessing apparatus including: an input unit used to inputchange-of-form information on a medical device; an image signalgenerating unit which generates an image signal from the inputchange-of-form information; an image information generating unit whichgenerates image information for acquiring a superposed image having athree-dimensional image of the medical device over a three-dimensionalimage a preoperative structure; and an output unit which outputsinformation on the superposed image.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram illustrating a system having a medicalimage display apparatus according to a first embodiment.

FIG. 2 is a schematic diagram illustrating a configuration of a medicalimage display apparatus according to the first embodiment.

FIGS. 3A and 3B are schematic diagrams of a curvature deformation unitof a medical device of a system in the medical image display apparatusaccording to the first embodiment.

FIGS. 4A and 4B are schematic diagrams illustrating the curvaturedeformation unit having an angle sensor.

FIG. 5 is a schematic diagram illustrating the curvature deformationunit having an attitude sensor.

FIG. 6 is a schematic diagram illustrating the curvature deformationunit having a pressure sensor.

FIGS. 7A and 7B are schematic diagrams illustrating that a part abuttedagainst the curvature deformation unit of an internal structure iscorrected to be enlarged by an image correction in a superposed image.

DESCRIPTION OF THE EMBODIMENTS First Embodiment

A medical image processing apparatus according to a first embodiment ofthe present invention includes an image information generating unitwhich causes a display apparatus to display a three-dimensional image ofa medical device over a three-dimensional image of a preoperative object(or an object before an operation and/or before an examination).

First, reference numerals in drawings will be described. Referencenumeral 1 denotes a medical device, reference numerals 4 and 401 denotecurvature deformation units, reference numeral 5 denotes a controller,reference numeral 6 denotes a navigation unit, and reference numeral 101denotes an internal structure.

FIG. 1 is a schematic diagram of a system including a medical imageprocessing apparatus according to the first embodiment. A medical device1 is inserted to an object 100 which is a subject human body during anoperation. The medical device 1 has an operating unit 3 and an insertingunit 2. The inserting unit 2 has a curvature deformation unit 4 at itsleading end. The curvature deformation unit 4 curves (or bends) to thedirection of in-plane as illustrated, for example, with aninternally-contained wire. When the medical device is an endoscope, thecurvature deformation unit may internally contain an image opticalsystem such as optical fiber and/or a treatment tool.

The medical device 1 is connected to the controller 5 through cables 8and 9. The controller 5 automatically controls operating conditions suchas an insertion depth, direction and/or angle of the inserting unit 2 ora position and/or direction of rotation of the medical device 1 itself,in addition to control over a deformation of the curvature deformationunit 4. The controller 5 is connected to the navigation unit 6 that is amedical image processing apparatus according to this embodiment throughcables 10 and 11. The navigation unit 6 outputs image information, and adisplay apparatus 7 receives it and displays the corresponding image. InFIG. 1, the display apparatus 7 three-dimensionally displays images of ablood vessel to be observed, a part of the inserting unit 2, and thecurvature deformation unit 4.

FIG. 2 is a schematic diagram illustrating a configuration of thenavigation unit 6 that is a medical image processing apparatus accordingto this embodiment. The navigation unit 6 has a preoperative image inputunit, an input unit which receives information from the medical device,an image signal generating unit which generates an image signal frominformation from the medical device, an image information generatingunit, and an output unit which outputs image information generated bythe image information generating unit to a display apparatus.

The preoperative image input unit receives a plurality oftwo-dimensional tomograms captured before an operation. The imageinformation generating unit segments the plurality of input tomograms togenerate a three-dimensional image.

A two-dimensional image captured before an operation here may refer toan image captured by an apparatus such as a CT and an MRI, for example.The image is an image of a wider area than an area to be displayed by adisplay apparatus. The two-dimensional image has in advance a markerthat is a coordinate reference for image capturing such that thecurvature deformation unit of the medical device may be displayed at acorrect position in a preoperative image.

The image signal generating unit generates an image signal on the basisof change-of-form information input from the medical device 1 to theinput unit. The change-of-form information from the medical device 1 mayrefer to tip position information, change-of-angle information,change-of-tilt information, and/or change-of-pressure information, forexample, on the curvature deformation unit 4, which may be acquired byinsertion of the curvature deformation unit to an object. From at leastone information piece of these information pieces input through thecontroller 5, the image information generating unit generates imageinformation for generating a three-dimensional image of a form of thecurvature deformation unit 4 in the medical device 1. The form of thecurvature deformation unit 4 may be a thickness of the curvaturedeformation unit or a contour of the leading end, for example.

Another information piece for identifying a form of the curvaturedeformation unit may be information known before the curvaturedeformation unit is inserted to an object, such as a structure of thecurvature deformation unit, an angle of a curvature deformation, and arange that may be captured by an imaging optical system if the curvaturedeformation unit has one. These known information pieces may be inputfrom the controller 5 to the navigation unit 6 or may be directly inputto the navigation unit through an input unit, not illustrated, withoutthrough the controller 5.

The image information generating unit generates image information fordisplaying an image (superposed image) having a three-dimensional imageof the curvature deformation unit 4 over a preoperativethree-dimensional image of an object. Image information on a superposedimage is transmitted from the output unit to the display apparatus, andthe display apparatus displays a three-dimensional image that is thesuperposed image.

The display apparatus may be a display for a PC, for example. Thenavigation unit 6 generates image information on a rotatable andtranslational image from a viewpoint requested by an observer.

Thus, the medical image processing apparatus according to thisembodiment may three-dimensionally display a figure of the medicaldevice over a three-dimensional image generated on the basis of apreoperative image.

The figure of the medical device to be displayed in the medical imageprocessing apparatus according to this embodiment is a form of thecurvature deformation unit of the medical device as described above. Thefigure of the medical device to be displayed in the medical imageprocessing apparatus according to the present invention may include bothfigures of the curvature deformation unit and the inserting unit. Thefigure of the inserting unit may be generated from information acquiredby insertion of the curvature deformation unit or known information,like a figure of the curvature deformation unit.

In a system having a medical image processing apparatus according tothis embodiment, the controller 5 may automatically perform operationson a position, a depth, an angle and/or a direction, for example, of themedical device itself, the inserting unit or the curvature deformationunit. On the other hand, in a system according to the present invention,one of those operations may be performed manually by an operator.

A system having a medical image processing apparatus according to thisembodiment may perform an operation on a subject human body as theobject 100. On the other hand, a system having a medical imageprocessing apparatus according to the present invention may perform anoperation on an animal as its object. A system having a medical imageprocessing apparatus according to this embodiment may perform anoperation on a subject human body as the object 100, and the medicaldevice may be inserted into the illustrated body cavity. On the otherhand, a system having a medical image processing apparatus according tothe present invention may be inserted to the body from the mouth, an earor the like or may be inserted to a lumen such as a gap in the brain, acolon, or a blood vessel.

In a system having a medical image processing apparatus according tothis embodiment, information pieces as described above may be moved(received and/or transmitted) through a cable or by radio between theoperation controller 5 of the medical device and the navigation unit 6.

In a system having a medical image processing apparatus according tothis embodiment, information may be moved through a cable or by radiobetween the navigation unit 6 and the display apparatus 7.

In a system having a medical image processing apparatus according tothis embodiment, information pieces as described above may be movedthrough a cable or by radio between components of the navigation unit 6.

FIGS. 3A and 3B are schematic diagrams illustrating a part (rigid unit)21 and the curvature deformation unit 4 at its tip of the inserting unitof the medical device 1 in a system having the medical image processingapparatus. FIG. 3A illustrates the curvature deformation unit 4 that isbending, and FIG. 3B illustrates the curvature deformation unit 4 thatis straight and is not bending.

The curvature deformation unit 4 has a plurality of node rings 41 in alongitudinal direction. A fulcrum 42 is provided between the node rings41. The curvature deformation unit 4 is curved (rotated) about thefulcrums.

The straight state in FIG. 3B according to this embodiment is an initialstate. The controller 5 recognizes the state as the initial state andrecognizes a state resulting from a curvature deformationtwo-dimensionally, that is, simply recognizes a form of the state.

From the state, the position where the medical device 1 is mounted (or atilt or a distance from an object, for example) and a marker on apreoperative image, the navigation unit 6 calculates the position of themedical device on a three-dimensional image for registration, and theimage information generating unit generates superposed imageinformation.

FIGS. 4A and 4B to FIG. 6 are schematic diagrams illustratingconfigurations of the various curvature deformation units 4. FIGS. 4Aand 4B illustrate a configuration in which the curvature deformationunit has a potentiometer and which provides change-of-angle information.FIG. 5 illustrates a configuration in which the curvature deformationunit has an attitude sensor and which provides information on a changeof direction or attitude. FIG. 6 illustrates a configuration in whichthe curvature deformation unit has a pressure sensor and which provideschange-of-pressure information.

The configurations of the various curvature deformation units will bedescribed with reference to FIGS. 4A and 4B to FIG. 6.

As illustrated in FIG. 4A, the curvature deformation unit has aplurality of node rings 41 and a potentiometer 441 at each fulcrumtherebetween. Each of the potentiometers 411 detects an angle of thecorresponding node ring as a value of resistance. FIG. 4B illustrates anequivalent circuit of three potentiometers illustrated in FIG. 4A. Thethree potentiometers 441 give voltages based on the corresponding anglesas change-of-angle information to the controller 5.

As illustrated in FIG. 5, the curvature deformation unit has a pluralityof node rings 41, and each of the node rings has an attitude sensor 442at a same position on its side. The attitude sensor 442 may be atriaxial acceleration sensor, for example. The attitude sensor 442 maydetect the direction of gravity in three directions applied to thesensor, that is, a tilt of the corresponding node ring (tilt of theattitude sensor) and give change-of-attitude information to thecontroller 5. Because the three-dimensional directions of the node rings41 is provided by the attitude sensors, the controller 5 may grasp theattitude of the curvature deformation unit three-dimensionally ratherthan two-dimensionally. The change-of-attitude information may also beuseful for generating a three-dimensional image fast and accurately bythe navigation unit 6 even when the controller 5 does not grasp theattitude three-dimensionally.

Because the three-dimensional directions of the node rings 41 areprovided by the attitude sensors, the attitude of the curvaturedeformation unit may be detected three-dimensionally even when thecurvature deformation unit only moves two-dimensionally as in thisembodiment. Thus, the attitude may be detected even when the curvaturedeformation unit 4 is compressed by an internal structure and is tiltedto a direction where it is not operable.

The attitude sensor may be an encoder, an optical sensor or the likeinstead of an acceleration sensor.

As illustrated in FIG. 6, the curvature deformation unit has a pluralityof node rings 41 and a potentiometer 441 and a pressure sensor 443therebetween. The pressure sensor 443 outputs a voltage in accordancewith a pressure. The pressure sensor may be a piezoresistive pressuresensor which is produced by an MEMS technology, for example.

The change-of-pressure information on the curvature deformation unit isgiven to the controller 5. On the basis of the change-of-pressureinformation, operation control information may be transmitted from thenavigation unit 6 to the controller 5. For example, when the controller5 inserts the curvature deformation unit automatically, the navigationunit 6 determines that the curvature deformation unit is in contact withan internal structure on the basis of the change-of-pressureinformation. How much the curvature deformation unit is to be retractedmay be instructed through the controller 5.

Second Embodiment

A medical image processing apparatus according to a second embodiment ofthe present invention is a medical image processing apparatus whichcorrects an image of an abutted part of an image corresponding to aninternal structure in a superposed image to enlarge the image of theform of the internal structure in the abutted part when the curvaturedeformation unit is abutted against the internal structure. Except forthis, the second embodiment is the same as the first embodiment.

FIGS. 7A and 7B are schematic diagrams illustrating three-dimensionalimages based on image information output by the medical image processingapparatus according to this embodiment. FIGS. 7A and 7B illustratesuperposed images in which images of an internal structure and acurvature deformation unit are superposed. Reference numeral 20 denotesan inserting unit, reference numeral 401 denotes a curvature deformationunit, and reference numeral 101 denotes an internal structure. FIG. 7Ais a schematic diagram illustrating a tip of the curvature deformationunit immediately before it is abutted against an internal structure.FIG. 7B is a schematic diagram illustrating an image corrected toenlarge the image of the abutted internal structure.

The internal structure and the curvature deformation unit may be abuttedagainst each other when insertion of the automatically controlledmedical device gets slower during an operation. Thus, when the insertioncondition changes, the fact is determined by the navigation unit on thebasis of the change-of-form information on the medical device, and theimage of the abutted internal structure (within a dotted circle) iscorrected to be enlarged.

More specifically, if the tip of the curvature deformation unit having apressure sensor in its leading part detects a pressure and it is knownthat an internal structure exists near there from a preoperative imageor a preoperative three-dimensional image before a superposed image, theimage is corrected to enlarge the image of the internal structure.

Thus, when the structure of an object to which the medical device isinserted is different from the structure grasped from a preoperativeimage in their sizes and/or positions, the image correction may providea more accurate image thereof.

Having described this embodiment in which a medical device having acurvature deformation unit having a pressure sensor at its tip, amedical image processing apparatus according to the present inventionmay have pressure sensors at a plurality of parts such as a tube partand a stick part of the curvature deformation unit, and an image of aninternal structure may be corrected to be enlarged from pressures on thetube part and/or stick part.

According to this embodiment, when the curvature deformation unit isabutted against an internal structure, an image correction is performedto enlarge the image of the contact part of the internal structure. Onthe other hand, a medical image processing apparatus according to thepresent invention may perform an image correction to delete or reduce animage of a part of an internal structure expected from a preoperativeimage to be abutted as a result of the insertion or deformation of thecurvature deformation unit if it is not abutted against the part.

As specifically described with reference to the embodiments, a medicalimage processing apparatus of the present invention may provide asuperposed image in which a three-dimensional image of the medicaldevice is over a three-dimensional image of a preoperative structure.Thus, an accurate three-dimensional image based on information may beprovided.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2012-131118, filed Jun. 8, 2012, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. A medical image processing apparatus, comprising:an image information generating unit configured to generate imageinformation for acquiring a superposed image having a three-dimensionalimage of a medical device over a three-dimensional image of a structure;and an output unit configured to output the image information, whereinthe image information generating unit corrects the structure in thesuperposed image on a basis of a state of contact, detected by adetector of the medical device, of the structure and the medical device,and wherein the output unit outputs the corrected image information. 2.The medical image processing apparatus according to claim 1, wherein, ina case where the contact of the structure and the medical device isdetected by the detector, the image information generating unit correctsthe structure in the superposed image in a manner that the structureapproaches the medical device.
 3. The medical image processing apparatusaccording to claim 2, wherein the image information generating unitperforms correction in a manner as to enlarge the structure in thesuperposed image.
 4. The medical image processing apparatus according toclaim 3, wherein the image information generating unit performs thecorrection in a manner that, of the structure, a portion corrected to beenlarged and other portion are displayed continuously.
 5. The medicalimage processing apparatus according to claim 1, wherein, in a casewhere non-contact of the structure and the medical device is detected bythe detector, the image information generating unit performs correctionto reduce visibility of the structure in the superposed image.
 6. Themedical image processing apparatus according to claim 5, wherein theimage information generating unit reduces the visibility of thestructure in the superposed image either by reducing size of thestructure or by deleting the structure.
 7. The medical image processingapparatus according to claim 1, wherein the detector is a pressuresensor; and wherein the state of contact is acquired on a basis ofchange-of-pressure information detected by the pressure sensor.
 8. Themedical image processing apparatus according to claim 1, wherein themedical device includes a curvature deformation unit.
 9. The medicalimage processing apparatus according to claim 8, wherein the curvaturedeformation unit has a serially aligned structure of node rings in anaxial direction of the node rings.
 10. The medical image processingapparatus according to claim 8, wherein the detector is provided on thecurvature deformation unit.
 11. The medical image processing apparatusaccording to claim 8, wherein the detector is an attitude sensorconfigured to detect attitude of the curvature deformation unit.
 12. Themedical image processing apparatus according to claim 8, furthercomprising: a control unit configured to control deformation of thecurvature deformation unit.
 13. The medical image processing apparatusaccording to claim 12, wherein the control unit communicates with themedical device wirelessly.
 14. The medical image processing apparatusaccording to claim 1, wherein the three-dimensional image of thestructure includes a marker; and wherein the image informationgenerating unit generates the image information by using the marker. 15.The medical image processing apparatus according to claim 1, wherein aviewpoint of the superposed image is changeable.
 16. The medical imageprocessing apparatus according to claim 1, further comprising: a displayunit configured to display the corrected image information.
 17. An imageinformation generation method of generating image information foracquiring a superposed image having a three-dimensional image of amedical device over a three-dimensional image of a structure,comprising: correcting the structure in the superposed image on a basisof a state of contact, detected by a detector of the medical device, ofthe structure and the medical device, and outputting the corrected imageinformation.
 18. The image information generation method according toclaim 17, wherein, in the correction, in a case where the contact of thestructure and the medical device is detected by the detector, thestructure in the superposed image is corrected in a manner that thestructure approaches the medical device.
 19. The image informationgeneration method according to claim 18, wherein the correction isperformed in a manner as to enlarge the structure in the superposedimage.
 20. The image information generation method according to claim19, wherein the correction is performed in a manner that, of thestructure, a portion corrected to be enlarged and other portion aredisplayed continuously.
 21. The image information generation methodaccording to claim 17, wherein, in a case where non-contact of thestructure and the medical device is detected by the detector, thecorrection is performed to reduce visibility of the structure in thesuperposed image.
 22. The image information generation method accordingto claim 21, wherein the visibility of the structure in the superposedimage is reduced either by reducing size of the structure or by deletingthe structure.
 23. The image information generation method according toclaim 17, wherein the three-dimensional image of the structure includesa marker; and wherein the image information is generated by using themarker.
 24. The image information generation method according to claim17, wherein a viewpoint of the superposed image is changeable.